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1. Introduction 


This document outlines a procedure for evaluating smoke taint in wines in an objective matter. The 
procedure has been developed based on the fundamentals of sensory science and quality control 
sensory methods. It adapts methodology that has been extensively applied with sensory panels at 
the Australian Wine Research Institute (AWRI), which produces robust results and is well correlated 
with chemical analysis. Although this is a rigorous method, some statistical power may be 
compromised as a result of the adaptation. 


The role of the regional association 


Developing a regional capability for smoke sensory assessment will support grape and wine 
producers and help them to identify and manage the risk of smoke taint. The regional association 
will be instrumental in deciding when a panel is required, maintaining a list of potential panellists, 
identifying a venue, scheduling tastings, providing administrative support, and conducting the 
tastings in confidence. 


The purpose of a regional smoke taint sensory assessment panel is to support producers who 
require sensory assessment of wines made from grapes that may have been affected by smoke 
exposure. In many cases the wines will be from small-lot fermentations made to assess the impact of 
smoke exposure. In assessing the risk of smoke taint, the current best practice advice from the AWRI 
is to use a combination of chemical analysis and sensory assessment. Chemical analysis can 
determine with a high degree of confidence if grapes have been exposed to smoke. Sensory analysis 
can help determine if the smoke exposure is likely to lead to a sensory defect in the resultant 
finished wine. 


The role of the AWRI 


The AWRI has developed a number of initiatives to support regions affected by bushfires, including 
ongoing support for the effective operation of regional sensory assessment panels. Please contact 
the AWRI helpdesk helpdesk@awri.com.au or 08 8313 6600 for assistance with conducting smoke 
taint sensory evaluations using the method outlined in this document, including panel screening and 
training, and data analysis. 


2. Sensory evaluation 


A trained, screened panel should be used, consisting of panellists known to be sensitive to smoke 
taint. The panel selection process is critical as each individual has different degrees of sensitivity to 
specific aromas and flavours. AWRI sensory data indicate that 20-30% of experienced sensory 
assessors (including winemakers) are not sensitive to smoke characters. The AWRI can provide 
assistance with conducting screening sessions for potential judges. 


Even with a sensitive, highly trained and experienced sensory panel, ‘false positives’ can occur, with 
non-zero scores given by assessors for unaffected wines, due to an expectation effect, which 


commonly affects assessors evaluating taints. Making judges aware that a clean sample is included 
in each set can prevent judges from expecting a level of smoke in every sample. 


Assessing smoke-affected wines can be extremely challenging, especially if it involves assessing 
small-lot ferments, which are prone to off flavours that are unrelated to smoke taint. Wines from 
small-lot ferments are often made from grapes that are sampled many weeks before optimal 
harvest. These wines may show overt green, unripe flavours and high acidity that can mask smoke 
characters. The ability to discriminate smoke characters from other wine attributes requires skill and 
training. 


Implementing sensory evaluation best practices allows reliable outcomes from sensory assessments 
of small-lot ferments, minimising the impact of biases and physiological effects on the results. These 
practices include: 


e independent judgements 

e blind sample coding 

e alimited number of samples evaluated in each session 
e uniform temperature across samples 

e proper rinsing and resting intervals 

e balanced random evaluation order across judges 
e minimisation of sensory fatigue and carry-over 

e inclusion of a negative, clean control 

e inclusion of blind replicate(s) or all wines repeated 
e avoidance of distractions 

e use of a structured evaluation form. 


This procedure has been developed to fit within the resources of a regional panel context. 


Judges 


There is significant variation within the population, including winemakers, irrespective of age, 
position within the organisation, level of experience or training to recognise smoke taint. 
Undergoing screening and training is an essential prerequisite for participating in a sensory panel. 


A minimum of 8 judges should be used, with 10-12 recommended. Using a smaller group would affect 
the robustness of the results, with the data obtained not as reliable or sensitive. All judges should be 
available and motivated to participate in the panel and have previously demonstrated the ability to 
recognise smoke characters consistently, through successful completion of screening tests (see 
appendix). 


Familiarisation with sensory methods 
Judges should be familiarised with the use of the rating scale by initially rating a set of samples, 


preferably actual smoke-affected samples, at increasing concentrations of smoke volatiles (four 
samples). Encourage judges to think about the presence of ‘smoky’, ‘medicinal’, ‘ashy’, 
‘ohenolic/tarry’ aromas or flavours, and avoid considering too closely bitterness, astringency and 


drying/‘harsh’ palate characters or ‘dirty’ aftertaste in the absence of ‘smoky’ flavour, as these are 
common reasons for false positives. 


The final choice of judges 
Judges should evaluate real wines with different levels of smoke taint as a practice before the first real 


sample set. Remind judges that not all wines are tainted, and there are usually clean control wines 
included in the assessment, so scores of zero should be expected in some cases. 


Present at least one set of six wines, with at least one sample in duplicate. Options for samples for this 
test include sourcing smoke-affected wines of varying intensity, together with a clean control, or using 
samples ‘spiked’ with smoke compounds. The AWRI can assist with providing samples and guidance 
for screening and selection sessions. 


Each judge's performance will be assessed for discrimination and agreement with the panel mean. 
Those judges who perform poorly will be either subject to further training or excluded from the 
panel. 


Sensory evaluation conditions 

e The tasting area should be odour-free, quiet, well ventilated and temperature-controlled, with 
no open fires or obvious smells. 

e Judges should face away from each other so as not to influence one another. 

e Samples should be prepared in a separate room and out of sight of the judges. 

e Ideally, assessments should be held mid-morning or mid-afternoon. 

e Judges should not eat or drink (including coffee) for at least 30 minutes before the test. 

e Assessments should be conducted in silence. 


e Atwo-minute rest is required between samples to minimise the carry-over effect from one sample 
to the next. 


e |f suffering from colds or other illnesses, judges should be excluded until they recover. 


e The AWRI will provide access to ShowRunner, an online electronic scoring system, to facilitate 
the sensory evaluation process and data collection. The software can be accessed via a QR code 
or an email link using the judge's own internet-enabled mobile phone, tablet or computer. 


Sample preparation 


Assign each wine (including repeats) a unique 3-digit code. Use data collection software or the 
University of Oregon's random number generator to generate 3-digit codes. Write the codes on the 
bottles to be poured. 





Code rows of glasses with a non-permanent whiteboard marker (see Figure 1) with the 3-digit 
sample code for each wine, with enough glasses for the number of judges, and align wine bottles on 
the bench with the corresponding coded glasses. 
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Figure 1. Setting up glasses for a session with six wines and 11 judges 


Ensure constant temperature across samples (e.g. 20-22 °C). If wines have come from cool storage, 


pre-pour them early enough to reach a uniform room temperature across samples (approximately 20 
minutes). 


Use disposable 30 mL plastic pourers (shot glasses), widely available from most supermarkets, to 
ensure a constant volume of 30 mL across samples. 


Use random presentation order. Each judge should receive glasses in a different order (not just 
starting in different positions of the same order) to reduce the impact of order effects. To achieve 
this, use the randomisation function of the data collection software, or, if not available, judges can 
shuffle samples randomly, or the person setting up the tasting can do this. 


Samples should be expectorated; spittoons are therefore required. 


Evaluate white wines before red wines. Where possible, split sets into groups of similar varieties or 
styles. 


Include negative and positive controls. The panel should be aware of the inclusion of controls and 
expect that not all samples will be smoke-affected. 


o A negative control is a clean, unaffected wine, preferably of the same variety as samples 
in the set and made in a similar way. A negative control must be included in each set. For 
example, if a set involves six aromatic white wines, followed by a set of three light reds 
followed by a set of four Shiraz, include one appropriate control in each set. 


o Inthe case of small-lot ferments made after a fire event, a commercially made wine 
from a previous vintage can be included as a control, but control small-lot ferments are 
preferred. 


o If the number of samples to be assessed is large and cannot be split over several 
sessions, then include at least two controls over the whole test. 


o A positive control (a known, obviously smoke-affected sample) should be included when 
possible, on a regular basis, to ensure judges are still able to recognise smoke 
compounds. 


Present a maximum of eight to ten samples per set (depending on the nature of the samples), 
including controls and duplicates, followed by a break of at least 10 minutes. 


Carefully consider the number of samples for the time allocated for the session (maximum of 20 
samples per hour). Twenty samples are recommended as an appropriate maximum number for a 
session. 


Include at least one duplicated sample in each set to assess judges’ repeatability. 


Assessing the wines 
Distribute devices with data collection software to each judge or ask them to open the session on their 


own devices. 


Ask judges to concentrate, be honest with themselves, not be too critical and not gravitate to the 
middle of the scale. 


Remind judges that there are clean control wines included in the assessment, and scores of zero 
should be expected. 


Remind judges to confirm the 3-digit sample code on the screen and follow the tasting order that has 
been individually assigned to them. 


Remind judges that all assessments should be conducted independently and blind (no knowledge of 
samples, regions, producers), with no discussion or comments made during the assessment. 


Ensure judges have a 2-minute break between samples to minimise carry-over effects and a 10-minute 
break between each set of samples. 


Encourage judges to rinse with water during the breaks between samples. 


An example of the electronic rating system appears in Figure 2, using mobile phone format. A line 
scale is used, with indented anchor points of ‘low’ and ‘high’ placed at 10% and 90%, respectively. 
Please note that a score of zero should be placed at the far left-hand edge of the scale, not on the 
‘low’ mark. This continuous type of scale expresses smaller differences and is, therefore, more 


discriminatory. Separate scales are used for rating ‘smoke’ and ‘overall fruit’ for each wine to be tested 
as well as an ‘other’ term, which may indicate the presence of off-flavours that could mask smoke 
taint perception. A comment box to specify the off-flavour is included to capture this information. 
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Smoke Taint Evaluation 


Please rate the intensity of the overall 
fruitiness and smoke taint if detected for 
samples 934 


If you do not perceive an attribute please rate 


it ZERO not less than ‘low' 


Sample: 934 


Overall Fruit 


| © Definition 
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Low High 


Smoke Taint 
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Low High 
Other 
Figure 2. Example of the electronic scoring rating template formatted for a mobile phone with the line 


scale and anchor points for rating the three attributes, ‘overall fruit’, ‘smoke taint’, and ‘other’ and 
attribute definitions. 


3. Analysis of results 


When using ShowRunner or other online sensory software, close the session and export the results 
from the software. Send the results as a *.csv file to the AWRI (helpdesk@awri.com.au) for analysis. 
A copy of the results will then be provided, presented as two tables (Tables 1 and 2). 


Table 1. Example of results provided by the AWRI, showing two judges’ rating of ‘overall fruit’, 
‘smoke taint’ and ‘other’ attributes for four wines (0-10). 








Judge Wine Blinding Overall Smoke Other 
Code fruit taint 

John Smith Control Chardonnay 413 6.63 0.12 0 
John Smith Chardonnay A 839 3.63 3.41 1.57 
John Smith Chardonnay B 899 1.53 5.82 3.7 
John Smith Chardonnay C 998 4.18 2.09 2.4 
Ana Silva Control Chardonnay 413 6.69 0.90 3.13 
Ana Silva Chardonnay A 839 3.86 2.64 0 
Ana Silva Chardonnay B 899 2.09 6.12 0 
Ana Silva Chardonnay C 998 5.78 1.11 4.32 





e Data is analysed using Analysis of Variance (ANOVA) with judge and wine as factors, for the 
attributes ‘overall fruit’ and ‘smoke’. 


e Inspecting the ANOVA results (Table 2), if there is a significant difference between the wines 
(indicated by a P-value <0.05 for the ‘wine’ factor) for the attribute ‘smoke’, a Dunnett's means 
comparison test is conducted at 95% confidence level using the clean sample as the control. (This 
is why it is essential to include a clean sample as a negative control in every set of wines tasted). 
This test will indicate which samples are statistically different from the clean control sample. If the 
mean attribute values for the test wines are significantly different from the clean control, a wine 
is considered affected. 


Table 2. Example of results provided by the AWRI, comparing each sample with the clean control 








, Smoke Significantly higher than 
Wine 7 
mean score control’? 
Chardonnay A 3.17** Yes 
Chardonnay B 6.3*** Yes 
Chardonnay C 1.5* Yes 
Control Chardonnay 0.7 - 





‘Dunnett's means comparison test. *** P< 0.001 * P< 0.05 


4. Results and decisions 


Sensory results should not be used in isolation when making critical harvest decisions. It is highly 
recommended that sensory results be considered in conjunction with corresponding chemical 
analysis of smoke marker compounds in grapes, including both the volatile phenols and glycosides. 
Ultimately, it is the role of the regional panel to provide information to producers so they can make 
the appropriate harvest decisions for their business. 


5. Motivation and panel performance 


After each session, the performance of judges should be monitored to ensure that performance 
criteria continue to be met. Judges’ individual agreement with the panel mean, ability to 


discriminate between clean control and different levels of tainted samples and repeatability in the 
duplicated samples should be taken into consideration. The AWRI can assist with panel performance 
analysis. 


Individual judges’ performances are assessed by checking the scores for negative and positive 
control samples and repeat samples. The panel’s precision is assessed by checking 
scores/conclusions on the repeated samples. Other checks that can be made include: 


o Comparing individual judge scores with the panel mean. 
o Looking at individual standard deviations from the panel mean when possible. 
o Looking for false positives; that is, checking if clean controls were scored low. 


o Checking smoke ratings for positive controls (known tainted samples should have high 
smoke scores). 


o Checking individual repeatability by comparing ratings for the repeated samples from each 
judge. 


Data from poor performing judges can potentially be excluded from the session, without reducing 
panel size below eight. If a judge consistently performs poorly, consider not inviting for further 
sessions and offer him/her to help with setting up and analysing results instead. 


It is important to appreciate and acknowledge the efforts that judges undertake to attend the 
sessions. Provide performance feedback to judges constructively, being encouraging and thankful for 
their time and effort. Keep individual performance results confidential. 


Judges should participate on a regular basis to maintain their performance. Weekly participation is 
desirable, with a minimum of monthly participation if possible. The AWRI will assist in re-training 
judges after long periods of interruption (six weeks or greater). 
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